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(54) PANORAMA IMAGE SYNTHESIS DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide the 
panorama image synthesis device in which plural 
images are synthesized in an excellent way, 
regardless of simple configuration. 
SOLUTION: An image picked up by an electronic 
camera 307 is copied to an HD of a computer body 
301 via a general-purpose interface 306 and 
displayed on a display device 302. Furthermore, the 
synthesizer discriminates whether an original is an 
original image mainly consisting of characters or a 
usual natural image after a corresponding point 
extract processing of the image relating to the image 
synthesis is finished and an overlap range of the 
image is obtained. Based on the result of 

discrimination, a way of seamless processing is changed to carry out the synthesis 
processing. 
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my^—^iy\- (JPEG^TIFF^) {Z^^y'^y 
y'-^^^m-r^ (S 1 0 0 3) « ^L-C. ^m;55j^^:> 
o/c^:7r-i'/i-^-i':/8 5 a^t;?8 5 b t^jEMi-^o 

[0 0 8 2] K8 7 aS.U?8 7 b ^^-<T 

10 ^-rs (SI 0 0 4) o ^^tu^x'^cCi^^m^ji. 51 
^(Dmw'T-^ ti.x^m-r^ (sioo8) « ^ir^w 

t;ijllil8X'<^T'~^l'S7"— ://n i o 8\cm^(Ov^'~ 
I D^ftJtTl&SU. ^ov^-^ I D^;^7^o^/7"— 
:://H 1 0 otc^^-rso 

[0 0 8 3] ^-^ilJ^X-^'5^#fi. C(7)x<y^-^ 

i-6 (S 1 0 0 5)„ ^nfiI3 8c7);^^o^7" — 
fci?'oXV>oX. ^/l—y I D(D^— K I D 8 9 a 
KI D8 9 a 
20 oXtTt>tv6o 

[0 0 8 4] 1^/1^— zf 

^f^^-r-S (3 1 0 0 6) o Cttfi. ;^^cri5^x— 
110 0 twMfctC^/l— I D 1 1 0 2 ^^It ^>'^ 
— 110 6. f^J5£0B#l 107. ^ fV-^Zt ^ ^ \ 

1 1 0^{^^'r-5o i//i-->^t5'-f >^l 1 1 Ol;i*i 
y-^m^e" <bf5SStt. fii^oMiiltSI'O^- K I D 

8 9a t)^^^isbhtl^. 

[0 0 8 5] 'lErL.X. ^^iIj^t'— ^ j:i@*ro 

x-^J' I D^f+ttx. 1 0 8<c^ftb. 

30 -r-^ I D 1 1 0 Siw^^-r-S (S 1 0 0 7) o 

[0 0 8 6] ^±(D:^'ryy'S 1000^^^51008 

CO— igoo^n^ri rsiij^^xtc^LxtT^o ^x 
(Dm^\^Mi.xmmi>mi:><:»ti^j^h^. ^^x=ie-u 
fct><7)CD4^x% y<y y-^mi^cDi^ji^-'^iimhthtit^t* 

0f)^^^^y^ (SI 0 0 9) . ^X^V^hthX^tl 

u^^ffli/^xmai-rs^^/ ^-^m^^^^^^ff 5 (s 

10 10) o cnt:iX^^^*^T-r5o 
[0 0 8 7] »C(w, tiTf2JX7^:y>^S 10 1 OtCioftS^^ 

40 y 7-=5'iii«'&^^31{-oi^x<7)SiW^'rSo 

[0 0 8 8] ia2 4ti. ^'^y ^-^^lJ^^^^^CO^^ — 
S5 1 7;i5^(D^Sgjf^^fT5i:9l;i^c^oXi>'5o 
[0 0 8 9] l^l3J;i*5l/>X. y<y y-^m&i^^^^S 1 

*^^^3.5/^"r6 (s 1 2 0 0) o 2^<ot^n. mm 

-tiyy/U^—h^^^miZA^ (SI 2 0 2) o 

^^^i^t^n. mm-r^^-^^^^m^^p^^ (s 1 2 

50 0 1) o wOJXT'j/y^S 1201. S120 2X<0-iett 



13 

ISC7ii>^^iii5/i5'i"e (S 1 2 0 3. S 1 2 0 4) „ CO 

-r^-t ^ h^^^^^^Tofc^ (S 1 2 0 5) .^10; 
[0 0 9 0] ^\Z^ mTfS;^v":y:/S 12 0 1 (;ijQtt'5:7 10 
[0 0 9 1] |^2 5fl. h'a'^^SI<D>'n— f^-^ — 

[0 0 9 2] l^l^{;l*5l/^-Cs ^-^mWi^^^ms 1 

7(i. ;^T. r;5»^^-^#;tfc^iM^>^.(?:»mfR^^^t 

-r^ (SI 30 D o -m-r^Mf&j^.^^on^ 

5 (S 1 3 0 2) o V-^m^t l^XUB-r^t 20 

^<OJV—/\^t UT. */M 0%. 0%:^—y<—y 

^^-7 5/:/-r6iGSf^iii 7(75^iii^i 5 0 i<Dm^Wt=^ 

y Tl 5 0 4t;i:^-rSQH<h/«^^o ^fc. :t-^<—'7y':f 

U-CV^5^tg^4<^$>^®H(i*®«l 5 0 2<r)«*^3iy 

r 1 5 0 b\z.7f^^mMt^j:^o Mi^^niy r 1504 

<D:xL])r^\Z^^^-^ly M 5 O 3 ^K-r >^ h 

fB(^/i— 5 t . M^S^i y T 1 5 0 5 y r *o 

1^-^ffiHi 5 0 6{;i7r:i-^y r4^i;i^rt;i-^.^.;i^s>^ 30 

So 

[0 0 9 3] :7n-^^- ht:iMS^. HfiiS;^^^ ,y:/S 
1 3 0 2X% i^Jl^Oi^-^i-S®H^^lwffil^-5>''^^^ 

(S 1 3 0 3) o r^. r(DM)^;,#.^^^i^S(7)i$^^cov^ 

(Nfi) \^J.±.t^^bf)^^Wm\^ (SI 
3 0 4) . ^Sffii^T<7)<t +5i'>^i?;:^s^iSbX*M. 40 

^S--ittf (S 1 30 5) o wt^^'&^S/'^^p^-^K^^ 
te'&fig^&JS^tTP (S 1 3 0 6) o :itoi¥»Bt>^^-r 

So 

[0 0 9 4] MI5;=^7^5':/S 120 2{C:JbttS >^ 

h'&^^a;SlwO^^T^0fft5g^■r5e 50 



#5|5p9-9 3 4 3 i 
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[0 0 9 5] |^2 6fi. :7/W:^— h^^^S(0:7P— ^ 

5 1 1 f)-^<D^mmv^^nb ^ 0 d/^foxi^So 

[0 0 9 6] 1^I^Iw*dI>X. ^-^S^-a-^^^S 1 

7(^. ^T. -7 5^^:^>5^®fflta:^^fT5;55 (si 60 
1) . ctb(itfrlB;?^x5/>^s 1 3 0 2 x*fc 

So 

[0 0 9 7]»c^;l. 4lHl^jS;.^am^SS:fT5c 

#;t^tLS{igmfi. 1 ^ i®^ 2 iiT. T 
-b. ;&:feco.4ii'9 XfcSo ^wX% C<^4o(0^ 

-g-tcoVNX^MS^fflttl^Sl^bX. ^in^ihnJt>M.t\. 

^^T^^/^^S 1 6 0 275^^ S 1 6 0 9^ 

X0^3gXfcSo 
[0 0 9 8] -^LX. ±(7)40(7):^X% (N 

•rS (SI 6 10) o t>b— ot>nffmf^-fe^;^-— 

^^H^-iltfo ^<D^X^ ^J^— I/-</U 

(^^t^ J:iNt>(D^Xc7){^m^#.XfeSir-rS (S16 1 

^^5>S«JUX^ofc^^fei7^cJ:9^^^/65^A^XVNX. jE 

attiUXb^5*a';55$>|9jj§^^ ^rX-. C<^>?.x:y>^ 
S 1 6 1 1 X% S^:7^ h UXiN5t>(7) (5F*&— ifeu. 
-</i-?&5*t>il5V>t(0) ^jlS^-rSJ: 5{w-rSo 
[0 0 9 9] ;j^7"5/:/s 16 1 lco^ll;55j^i^S^. ^ 

(D^f&^<=7 (s 1 6 1 2) . m^^f&^m 

(S 1 6 1 3) ---tmtsf)^. CtLfiMfS^T":y:/S 1 3 

05. si306^ m—^mx^ t) . mm-x^m-r^. 

[0100] m^^'^'r-y'Z^S 1 2 0 5{;i4bMtS'fe 

^^—v^f^^m.\z^\^^x(Dmm^T^o rrxco^n 

[0 1 0 1 1 12 2 7 fi, -fe^^h— h^^^lleO:7t3— ^ 

5 1 7.;55^(?D*0fSIS3f^^?T3 J:5J-*oXV^So 
[0 10 2] |^|2{C:fcnNX. ^•^ilj^-&^^li5 1 

git«^Sl^-rS (SI 70 1) o ^x^x-^y^iy-^m 

m^wt-^-r^t^ (s 1 7 0 2) . crx*<DiSHfi. 

OffiH (ffl>^^ixS::i--i^*cO'&i^^fcficg(D^S®H + 

[0 10 3] y^y'y-y^S 1 7 0 2 co^S;&5^^pS ^ . 
<0>^JC,^.fflif5^il (S 1 7 O 3) . -g^^/^^^-^^^ 
(S 1 7 0 4) . (S 1 7 O 5) 



(9) 
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[0 10 4] MJt^j^^m^m\^o\^^x(owi^Mi:'r 
10 10 51 ^-r. mis \^xMJt^j^iAm^m(D 

[0 10 6] rrX'^lSJi. Mft^.^Mm(Dm<D&t:^ 

mt i^xi^m^m^m cx^^o lo 

[0 10 7] $-r. WLBm(D/U--/U\Z<Oot^. "ri^y" 

u-hi:wt^T^mm2 0 o ^i-x. ms o%mi o%(o 

^5ff^E■r-5^|g^4^0$5H,^rv^5 w^r% ^10 0%. « 

^T^^^T'u— h®«2 0 0 3i: LX^H^ffi-To ^KD^ri^ 
>^l-'- hlj«2 0 0 3^. 1^— ^ffiSffl2 0 O 4 JiiCBV^ 

^^5$:. 1^-^®H2 0 0 4±^l@j^-ro-r^UT 

dS. ^^«lJ^JlTX$>iXf^. ^(T^/K-rV-hl^i (x, y) 
t (x' , ) ^Mft^.^.(D^rtL-Xi^^'r:5o 

[0 10 8] ^±(Dmm^^^M}&.^mmmm(^m.mt^j:^ 

[0 10 9] ia2 8j:j:. Mft^M.i6m^m(Dyt2-^^- 
h^^i"t<3ox*>5o i^x 13 2 8<?:):7ci— 

[0 110] i^iat;i:joiNX> ^-^y ^-^p^^^^^s 1 
7fi. ^-f. ^s/v^tttap^^f^^-r-s (s 1 9 0 

1) o -^UX. rOn:o,o;?^4^|U^t^f7:,y^V:7^l— 

»:^t-^®H2 0 0 57&^^. ^iyv^;^5@f^^ii[J!^±5Sv^2K 
>f Vh^rS-r (S 1 9 0 2) « -?rUX. ^<7)jK><>^hd5 

^riri. ^(D7i^-( hf)^ibmm.± nmm-r-o<Dm^xm 

^^«3!9fflU7">>^U— hiiJ^^^5 (S 1 9 O 3) « 

[0 111] »:(w. ^<7);K-f >^h(7)firB^>ib. 
tOf--^©H?rK;t-r5 (S 1 9 0 4) o ^UX. 1?— 40 

(S 1 9 0 5) o 

[0 112] ^fc. :L(Dm^(D^i^m.^K ^ti^x(Dm 

A^m^^bf)^%:^:^y^ b (S 1 9 0 6) . ^oX^th 



Sr^fiF-re (S 1 9 0 7) o gAJi^ri^-^^H^Xtc^ 

[0 113] ^ux. i^-^|5H^x^i^-^ufc7:)-^ 

:x,>y^l. (SI 9 0 8) . ^<7)jg;^5}t4^>bttfc*/hfit;d^ 
^it^UXW»T-r6 (S 1 9 O 9) . 0f^1fiLJ:«9/h^ 

f^wufcTK^rv-hc^^nis (x. y) 
tirc:^^:^h<Dmm (x' , y' ) ^(Dm^i^m.(Dm 
^^m-r^ (s 1 9 1 0) o 

[0 114] ^±^'r>y^l — h^^«&H:^giSlC^UX 
(S 1 9 1 1) . J^TUfc^Mlt^oy ^ h't'O:^ 

x(Dm:^M^f)^h^(D^i^m.^:^it>. :ith^—wii^^/^m. 

t L.XU^'T^ (S 1 9 1 2) o l^XJiX>^JS.^.am^E0^1l 

[0 115] ^^/"^7^-^K:^^^(col^TI^fc 
BJi-6o iij^^2tt:i: Ufci<lr#l;i (2«cil^±o^^co^ 
2^(D^fS.(Df^^Ml.fj:<DX\ ^-ril2tft:X%;t 

(x, y) . (x' , y' ) (iJgJlTOJz^JC^^^o 
[0 116] 

mi] 

fx'\ fj^cosB smeY^ 

^ in(cos6*x^rine«^-*Ax) ^ 
^in(— sin 6 -x + cosB • — 



3-01 



XJ9f 



O^iS.^.ttmMSXH. ^JS,^. (x, y) . (x' , 

[0 117] i-;fefc>*>. 

£ = 2 [ { (Ax+By + C) -x' } 2+ { (-Bx 

+ Ay+D) — y' }2] i n 

0^i4=X% 

[0 118] 

[|S2] 



(10) ^#^sp9-9 34 3 1 

17 18 

as/ dC- (Jx)^ H^y)B^ »c-(2x-) - 0 
ae/ aD - -(5;x)2? + fiD .(Jy ) - 0 



10 119] C C -C% 

PI =£ x2 +£ y2 

P2 =Z X 

P3 =Z y 

P4 =£ X x' +S y y' 

P5 =^Z X y' —S x' y 

P6 =S x' 

P7 =L y' 

P8 =n mJt>J^.(Dm) 

[0 12 0] 

Pi+P?-flA 
^ P6-P^^-P»g 
Pt 

P« 

rtO^N'^T^-tJ^P l;5>bP8^*de). ±iS^{cfA^Ai-'5 

[0 12 1] m^. m^^j&^m\^-o\^^xmm'r^o 

[0 1 2 2] 132 9*i. ^]&^m^{^(0^^-^-r- h 
^^i-t<D-Cfe'5o fSSi. ^2 9(75:7 13— 

[0 12 3] [^i3(3:*5i>T. '7^mm.^m.^ms 1 

no (S 2 9 0 1) „ 
[0 12 4] 7:x:?'®H^0S^tCOl>T 

x' =Ax+ByH-C 
y' =-Bx+Ay+D 

[0125] mx,t^m 1 9 Jw^i-J: 5> lw^tw{i@i-6i® 

«3 0 0 4dS6 4 0 X 4 8 0 K5/ v<Dm^t-r^t. m 

^ (6 3 9, 0) (6 3 9, 4 7 9) S:±ft* (x. 



y) \CitAl.Xmht^ti (x 1' , y 1' ) . (x 

2' , y2M :^(DmiMiS O 0 3(D::t—y<—^ ^yzf 

mmt^j:^o ^^'y^jEHs 0 0 2*5 

^^aOOlT&N o/^^'-g-^p^^^^^-i: LT3l^Je)^);lx 
t5H(0 **75^ ^ ^ C i?>S:3t $ tt^cW 3 0 

t>oX®*K3 0 0 3 

[0 12 6] ^^Cl. ■X^±i$^(Dm^fj:(D:^\ ^ 

<^:>*^^m'J-rs t ;^ h^/^A^H^trp (S2 90 

2) o 

20 [0 12 7] ^-r. ^^;/:7^®HF^3 0 0 2 0® 

M<r^tfo 

[0 12 8] ccx% 13 2 o:s.r/i3 2 1 {z.m&<D\i:^ h 

1. a 2. a 3 b 1 . b2. b3^:^i65o 

L-C. b l*su#i>ii:t h 1 J:!9:^#< (:S:^<i:St?;h. 
^^^^~'^M&s±'^AyXh^X) . b 2;65L#l><t t h 
2 J: «9 /J^ ^ < (^^*^x xh^^^^t S ^ 6 ^ c?5 
i^(Di!)>-''^m^^<P'^j:<) . b 3;65Ul/>#ia[t h 3 
(3K^S^tt^gP5)^^-SSJ!^AJi-g-Ayr-v>-5) 

[0 12 9] ±m<oXo\:^l^x:^Tiy:fs 2 9 0 2Xt 

f)^t*0f)^m^l\^ (S 2 9 0 3) . CO^i^JSU^^lilcto 
T. v^— AU';^^S^'a^/<ei^^^^ll (S2 90 
4) . X(l'>-^l^;=^^S^^tr&J*^S^m^ (S2 
9 0 5). ^ll^«i^Ti-5o 

[0 13 0] ^{z^ mU^^y'iyys 2 9 0 4 0>'— A^' 
[0 13 1] ^T. 02 2^#.^.UT'>— AUXftlS^ 

[0 13 2] ^2 2J::citt^l3^Ufctj<o^:^i-, il^ 
50 ;&5:feiii«2 10 1. 2 10 2 :fetii«2 l 



(11) 
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i-5o z(D^]$.m«kmm2 i o str. i^'r&mm2 1 o 

1 ^o/^t'-g^fc^^tT^^^-f 0 0 1 ^X*0®^4:^CD^ 

[0 13 31 ^^mm<om^<Dm^ (x, y) |;i 

x' =Ax4-By + C 
y' =-B x+Ay H-D 

MJt?-r^ (x' , y' ) ^^if>^o ^UT*!® 

(x' , y' ) (x, y) r6o 10 

[0 13 4] 03 0*^. i/— 

'&^^cQi^<D:7n— ^-Y— h^^-r>b(OX^6o r^. HIS 

y ^-^m^-^^^©: 5 17 ;65^<0^1li«jf^^n 5 J: 5 
[01351 ^-fs ^ 1 (DMiMi (1^ 2 2 TtO:feilJ^) CO 

2^so®«^^^iij««g*ac^ x^xm^-r^ (s 2 2 o 

1) . -^ErbT. mi(Om^<Dy4 ^'S 0 0 1 X^&m(D 20 
«S«^::o-g'^ili^«S«£l;im(j:=j t'— (s 2 2 o 

2) c 

[0 13 6] »:tc. -&fiKiij«(0^ D (x, y) 

x' =Ax+By+C 
y' =-Bx+Ay+D 

Mft^-t^ (x' , y' ) (S2 20 

3) o -lErUT. (x' , y' ) fi^2(0®« (132 2T- 
<7)*lJ«) l^(3:$>-5;!l>^5*^^f"^iyi5' U (S 2 2 0 

4) . /^tttLJ^. ^aiC7:)>?^7"^/:/S 2 2 0 6--^^^it 30 
htli^s (x' , y' ) (Omm^ (x. y) {C=if 

— i-^ (S 2 2 O 5) o 
[0 13 7] ^^{C. £Jl±CO:Xv"y:/S 2 2 0 37!i>^S 

2 2 0 5^ x(Dmm^^^mm(om v (Dm^±x \zm l 
xm^Mi^notLf>^t^^t)^i:mmi^ (S2206) . 

[0 13 81 2^t:i. mm:^Tiy^S 2 9 0 5(Oi^— Au 
;^ iiO;S «r-^ tf M^-S^fife^Oill I Co I > X U& "T S o 
[0139] 031 (i. i^-J=>^i-^^mi:^t^miSi^^ 

-^tiife-B^^£^©5 1 7if^-^<DmmW}ilF^rr^Xo{^f^<:> 

[0 14 01 ^-r. ffrlB^^T^^yy^S 2 2 0 1 

^ 1 (omiSi (0 2 2 T'co;feiij^) 2 iS(omm^^f&m 

«ffl«c^ UT5t»-r6 (S 3 2 0 1) „ AU- 

:^^m^n 0 mm 3 o o 3 j: i^tEm(Dm^^'^&m^ 
mm^^m\^=^ e— r-s (33202) « 

[0 1 4 1 1 ^LT. >— ^UXftlS^tTP (S 3 2 0 

3) o -r>fch»*>. ^-ris* 50 



9-93431 



20 



= Ax + B y -fC 



y' = — Bx+Ay+D 

*p Is p 2^7kisb^o ^UT. 

^fimm p3= (l-a) *pl4-a*p2 

[0 14 2] ^7">>>^S 3 2 0 3X->— AUX^^^JjSl. 

jt'S^. m2(DmiSL(Dm^<ommi:^^mi^m^ 

i^irie-u (S 3 2 04) . ^n^j^T-r^o r^. 

-^!9*fix 03 0<7);^7",y:/S 2 2 0 375^P3S 2 2 O 6 

^m^^i^^-r^ r ^ ^5x# 6. 

[-01431 Jl^±U£P>^ U^c J: 5 \Zy^Mm<DJ^m\^^ti 

x^\^^n^2m^i.(Dmm%:m»:'r^:iti)*^xt^ y< 
[0 14 4] im2(Dmm(Dmm) »:tw. ^i^Bjmj;^ 

[0 14 5] JeJLT(;i»2(D^JS(0?l^ffi^ ux. 

^^zfim^oy^^xmrn-r^. 

[0 14 6] f^. ^(7)^2(7)||JS(OJF^|iX-C7)«^Slt/Sb 

m^m^mi(o^m(Dmmtmmx;ho. mi 9 

X(0 3 o 0 l(0^fj:€^t>'^(0^^>'%:^^^t::L^td 
\-ff)m^j:^o J:oXwCX{i. ^l(0^iB<^?fl?ffiirf±S 

[0 1 4 7 1 HfrlE^ 1 (D^j6£(OJg^X(i^V 7-^-^^^ 
— V^yy(D^<Dn^'i%^^^^t^'^(Dm2 S0 2<D^ 

St. 4#ffi:.*C<7)^/j:i/>7>r^^2 8 0 14:^*5^. ^ 

So 

[0 14 8] 2fet;i:*:^MOifF^^X(7)i^f^<S:I^B.^-r5o 
[0149] 033 \z:^mM(DmmX(Df^Wi^^£'^^(r> 

li^tl^^(c%;tXJ:v\ 

[0 15 0] is]0tcioi>x. y<y ^-^mi&^f&^&s 1 

( S 3 3 0 1 ) o ^ bX. 01 9C03002 X?r:-r 

h±x<Dmmi:nm-t^ (s 3 3 o 2) o 

[0 1 5 1 1 ciT.p>^o®*|io^fp^*j?)^iS:s 
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umClftA-r^ {S 3 3 0 3) o i^CCT% M^^-tX^ 

[0 15 2] ^LX. ^SCS um^e^^^^ma x J: "P^^ct 10 
\,^f)^t^^f)>mmi^ (S 3 3 0 4) . ^IgcS um^i^^^l^m 

fj:^. ^3SSum^^»ma x(;iftAUT»i^-r-5 (S 

3 3 0 5) o ^fc^-f V-cT^ffiMp o s hi^l^-T^o 

[0 15 3] ^±^:^-y<'-v iyy'mm(Dmh^m'^x 

131 9T*0^-r :^3 0 0 1 t LT^^U. I^^CO^H^ 

[0 15 4] ±,m(oxo^^m2(Dm:M.<Dmm\^^t\^^. 20 

[0 15 5] (B 3 co^iSc7)?f^ffi) »:t:is :^^m\^X,^ 
[0 1 5 6] J[^Tlr^30^:^(7:)JF^^<^ UX. tJ^^feOv- 

[0 15 7] j^. w(Dm3 0^JS(7)?F^^X-(7)1f^25:U^®J 

ot::cx*i. ^i(D^i£(0?F^ffi^ji^/^6lfR5>twOV> 
[0 15 8] :^MM<Dmm{zx^{t:^^i.^mn. y<y'^ 

[0159] -r^jify-h. ^mm<ommxi^. ms4 4o 

-fJz^t;!. mi(DlJ^3 5 0 3fi^O^^^T^-^J*ilj 
fe(J0-Sl5<l: LTffil/>> ^(7)«Si55>3 5 O llil^UTtr*^ 

<DZ7^/U^ ti.Xn. ^^ftms SiZLTjk-tXo^JiS^ 3 

[0 16 0] m^. :^mM<rmm\z.^\i^9s^^w^-r 
[0161] 036 \^^mM<Dmmz,i6\ci^m^^^^ 

[0 16 2] |^0^C*5^^T. 7-^iij»'&^^©5 1 50 



4#M^9-9 3 4 3 1 
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^-f. Si Oia* (13 3 4 T*c?:>:fe®«) 2 fgO 
^tS^^^Ufifc®*^ UTm«-rS (S 3 4 0 1) o ^ 
UT. Bi<7)iii«*r(0-&^lli«ffli«(;i»*ifi«2:=it-— t- 

5 (S 3 4 0 2) „ 
[0 16 31 ^^m^(On'0 (OW^ (x, y) d 

x' =Ax + By+C 
y' = — B X -i- A y 4-D 

tcj:!p. (x' , y' ) (S34 0 

3) o ^Ur. (x' , y' ) fl^2(Olj^ (0 3 4T* 
0:&®^) P^{;i^6^^<if U (S3 4 0 

fe. fctLf^ (x' , y' ) <r>mm^ (x, y) iCnfc'- 
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JPO BJid NCEPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2 **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)l 

[Claim l] The panorama image synthesizer unit characterized by to have a distinction 
means distinguish whether it is the image with which the image of said dupUcate image 
field makes an alphabetic character a subject in the panorama image synthesizer unit 
which compounds two or more images which have the image field which overlapped 
mutually, and creates one synthetic image with a large field angle, and a synthetic means 
perform said image composition after performing a different image processing according to 
this distinction result. 

[Claim 2] The panorama image synthesizer unit characterized by having a synthetic 
means to compound two or more images which have the image field which overlapped 
mutually, to add shading"off processing to the boundary part which connects said two or 
more images in the panorama image synthesizer unit which creates one synthetic image 
with a large field angle, and to perform said image composition. 

[Claim 3] While compounding two or more images which have the image field which 
overlapped mutually and creating one synthetic image with a large field angle When the 
binary printer printed by the error diffusion method as said printer is specified in the 
panorama image synthesizer unit printed by the printer which had this synthetic image 
specified The panorama image synthesizer unit characterized by having a synthetic means 
to perform said image composition bordering on Rhine where the brightness of the image 
in said duplicate image field is the highest. 

[Claim 4] It is the panorama image synthesizer unit according to claim 3 which has a 
distinction means distinguish whether it is the image with which the image of said 
dupUcate image field makes an alphabetic character a subject, and is characterized by for 
said synthetic means to perform said image composition after performing a different image 
processing according to the distinction result of said distinction means, when the binary 
printer printed by the error diffusion method as said printer is not specified. 
[Claim 5] Said synthetic means is a panorama image synthesizer unit according to claim 1 
or 4 characterized by performing said image composition without performing seamless 
processing to the image of said duplicate image field when said distinction means was the 
image which does not make an alphabetic character a subject and it distinguishes, and 
performing seamless processing, when it was the image which makes an alphabetic 
character a subject and distinguishes. 
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[Claim 6] Said distinction means is a panorama image synthesizer unit according to claim 
1, 4, or 5 characterized by the image of said duphcate image field distinguishing an 
alphabetic character based on the luminance distribution of an image [ in / for whether it is 
the image made into a subject / said duplicate image field ]. 

[Claim 7] Said synthetic means is a panorama image synthesizer unit according to claim 1, 
4, 5, or 6 characterized by not perform^ing seamless processing to the image of said 
duplicate image field when said distinction means was the image which makes an 
alphabetic character a subject and it distinguishes, but performing said image composition 
bordering on Rhine where the brightness of the image in said duplicate image field is the 
highest. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention compounds two or more images which some images 
overlap, and relates to the panorama image synthesizer unit which creates one synthetic 
image. 
[0002] 

[Description of the Prior Art] Two or more images which some images overlap (duplication) 
are compounded on a computer, and, generally processing in which one synthetic image is 
created is called panorama composition. This can be called processing from demand of 
wanting to photo a wide image and to make it the image of one sheet. Moreover, in an 
electronic camera, as demerit in comparison with a film-based camera or a scanner,, 
although the lowness (little of a pixel number) of resolution is pointed out therefore, it is 
important [ panorama composition ] for the image photoed with this electronic camera also 
as a means which photographs a high resolution image [ photograph / not only / a wide 
image ]. A manuscript, a magazine, etc. of a sheet are divided into plurality, and are 
photoed, and, specifically, power is demonstrated for it to divide scenery into plurality and 
be wide, and take [ acquire about the same high resolution data as a scanner, and ] a 
photograph to high resolution. 

[0003] Here, in panorama composition, the processing which blots out the knot is 
important and the grace of the done synthetic image is influenced greatly. Processing (it is 
henceforth called "seamless processing".) which blots out a knot as shown in drawing 38 as 
general technique is performed. That is, in the part to overlap, according to the location of 
a pixel, a synthetic ratio is changed gradually, and it adds so that both images may become 
by a unit of 50% in the location of middle. When overlap is large, as shown in drawing 40 , 
it is predetermined width of face, and seamless processing is performed. 
[0004] This seamless processing is effective in especiaUy natural drawings, such as scenery, 
and the high image of grace without a knot is obtained. 
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[0005] 

[Problem(s) to be Solved by the Invention] However, there were the following problems in 
the above Prior arts. 

[0006] Although the synthetic location of the image overlapped using the approach of 
extracting corresponding points, among two or more images to compound is decided in the 
case of panorama composition, an error is somewhat included in a synthetic location at this 
time. That is, the minimum extract precision at the time of the extract of corresponding 
points is 1 pixel, the precision not more than it cannot be guaranteed, but, therefore, a gap 
of less than 1 pixel produces it as a synthetic location of a pixel. 
[0007] Moreover, in the case of the image photoed with the electronic camera etc., 
distortion of some of images has produced the image of the part located in the periphery of 
a lens, and a gap is included in it also in this reason. 

[0008] On the other hand, in the image with alphabetic characters, such as a manuscript, 
the contrast of the white of paper and the black of an alphabetic character has clarified. If 
this manuscript is compounded and carried out and seamless processing is performed, each 
is transparent and is seen by the alphabetic character of the part which performed 
seamless processing as shown in drawing 25 , and since the contrast of an alphabetic 
character is high, that gap will look notably. However, since it is more desirable to be low 
connected smoothly by contrast compared with the alphabetic character image in the case 
of natural drawing, seamless processing is effective. However, about a thing with 
alphabetic characters, such as a manuscript, the opposite effect is induced as mentioned 
above in many cases. 

[0009] Then, it is made in order that this invention may solve such a problem, and though 
it is a simple configuration, it aims at offering the panorama image synthesizer unit which 
can compound two or more images good. 
[0010] 

[Means for Solving the Problem] After performing an image processing which is different 
fi'om a distinction means distinguish whether it is the image with which the image of said 
duplicate image field makes an alphabetic character a subject in the panorama image 
synthesizer unit which this invention compounds two or more images which have the 
image field which overlapped mutually, and creates one synthetic image with a large field 
angle, according to this distinction result in order to solve the above-mentioned technical 
problem, it is characterized by to have a synthetic means perform said image composition. 
[00 ll] In addition, said synthetic means performs seamless processing to the image of said 
duplicate image field, when said distinction means was the image which does not make an 
alphabetic character a subject and it distinguishes, and it may be made to perform said 
image composition without performing seamless processing, when it was the image which 
makes an alphabetic character a subject and distinguishes. Moreover, you may make it the 
image of said duplicate image field distinguish said distinction means based on the 
luminance distribution of an image [ in / for whether it is the image which makes an 
alphabetic character a subject / said duplicate image field ]. 
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[0012] Moreover, this invention is characterized also by having a synthetic means to 
compound two or more images which have the image field which overlapped mutually, to 
add shading-off processing to the boundary part which connects said two or more images in 
the panorama image synthesizer unit which creates one synthetic image with a large field 
angle, and to perform said image composition. 

[0013] Furthermore, while this invention compounds two or more images which have the 
image field which overlapped mutually and creating one synthetic image with a large field 
angle When the binary printer printed by the error diffusion method as said printer is 
specified in the panorama image synthesizer unit printed by the printer which had this 
synthetic image specified It is characterized also by having a synthetic means to perform 
said image composition bordering on Rhine where the brightness of the image in said 
dupUcate image field is the highest. 
[0014] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is 
explained to a detail with reference to a drawing. 

[0015] (Gestalt of the 1st operation) The gestalt of the 1st operation by this invention is 
explained first. 

[0016] Drawing 1 is the outline external view showing the example which applied the 
panorama image synthesizer unit by this invention to the personal computer. 
[0017] As shown in this drawing, this equipment has the body 301 of a computer, with the 
display 302 which displays the various data containing the image concerning panorama 
composition respectively connected to this, the mouse 303, and the keyboard 305, possesses 
an electronic camera 307 fiirther and is constituted. 

[0018] Here, a mouse 303 is a typical pointing device and is equipped with the mouse 
button 304. 

[0019] Moreover, an electronic camera 307 has the internal memory which records the 
information about the image which self photoed, and is connected to the body 301 of a 
computer by the general interface 306 in which an image transfer at high speeds, such as a 
bidirectional parallel interface and a SCSI interface, is stiU more possible. And an 
electronic camera 307 is set to panorama image photography mode when photoing the 
image applied to panorama composition apart fi:om the time of the usual photography. 
[0020] Drawing 2 is the block diagram showing the configuration of this equipment 
containing software and hardware. 

[0021] said " drawing - being shown " as - a book - equipment " hardware • 509 this " 
hardware " 509 " a top " operating -- an operating system " (- OS " 505 " this " OS - 
505 " a top operating - application software - 504 having " constituting - having " 
**** . In addition, although naturally contained as requirements for a configuration among 
the blocks which constitute hardware 509 and OS505, when explaining the example of a 
gestalt of operation of this invention, it is not illustrating about the block which is not 
needed directly. As an example of such a block that is not illustrated, there is a memory 
management system etc. in CPU, memory, and OS505 in hardware 509. 
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[0022] Here, OS505 possesses the input device managerial system 506, the drawing 
managerial system 507, and the file system 508. 

[0023] The input device managerial system 506 has the function to enable it to receive a 
user's input, without application software 504 being conscious of hardware 509. 
[0024] The drawing managerial system 507 has the function in which application software 
504 enables it to draw without being conscious of hardware 509. 

[0025] A file system 508 has the function the application software fair 504 enables it to 
output and input a file, without being conscious of hardware 509. 

[0026] Moreover, hardware 509 possesses a keyboard interface 510, the mouse interface 
512, the video interface 513, the disk lO interface 514, the hard disk (HD) 515 that stores a 
file and data physically, and the general interfaces 306, such as a bidirectional piEuraUel 
interface or a SCSI interface. 

[0027] Here, the disk lO interface 514 is for reading and a file system 508 to write HD515. 
[0028] The video interface 513 is for the drawing managerial system 507 to draw on a . 
display 302. 

[0029] A keyboard interface 510 is for the input device managerial system 506 to receive 
the input of a keyboard 305. 

[0030] The mouse interface 512 is for the input device managerial system 506 to receive 
the input firom a mouse 303. 

[0031] An electronic camera 307 can be connected to the body 301 of a computer, an 
electronic camera 307 can let the input device managerial system 506 pass by this, and a 
general interface 306 can exchange image data etc. now. 

[0032] Moreover, application software 504 possesses the image data management system 
501 with the data control means 502 and the data display means 503, and the panorama 
image composition means 517. 

[0033] Here, the data control means 502 is for managing image data by attribute 
information or the kejrword by a user's input. 

[0034] The data display means 503 searches and expresses the image data managed as the 
attribute information or the keyword by a user's input. 

[0035] Moreover, the panorama image composition means 517 registers the image of the 
result of performing the image photoed in panoramic exposure mode through the image 
data management system 501, and having compounded panorama image composition 
processing for it further in reception and three kinds of synthetic modes (full auto 
composition, auto composition, and semi auto composition) mentioned later, to the image 
data management system 501. 

[0036] The function as a distinction means to distinguish whether it is the image with 
which the image of the image field where the panorama image composition means 517 
furthermore overlapped makes an alphabetic character a subject is realized, and further, 
after performing a different image processing according to this distinction resxilt, the 
function as a synthetic means to perform said image composition is realized. 
[0037] The panorama image composition means 517 as a synthetic means compounds like 
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drawing 3 without performing seamless processing, when judged as an alphabetic 
character subject by the distinction result. That is, it has come to stick an image bordering 
on Rhine 2601 of the middle of an overlap location. Although some gap is produced in the 
stuck alphabetic character part by this technique, since the amount of the part which has 
shifted becomes less, compared with the result in seamless processing as shown in drawing 
40 of the conventional example, it serves as a synthetic high image of quality more. In 
addition, when not belonging to an alphabetic character subject, the usual seamless 
processing is performed. 

[0038] Although it distinguishes that it is that the image to compound belongs to an 
alphabetic character subject, the panorama image composition means 517 as a distinction 
means takes the histogram of the brightness of an image, and distinguishes distinction of 
being that this image to compound belongs to an alphabetic character subject. That is, if it 
seems that it judges according to luminance distribution as shown in drawing 4 and 
drawing 5 , for example, luminance distribution is distributed over the whole like drawing 
5 , it will be judged as natural drawing. Here, the range which takes the histogram of 
brightness is good in the range which an image overlaps, and since it is not necessary to 
cross to the whole image, improvement in the speed of processing can be attained. 
[0039] Next, the DS of the information about the image which the electronic camera 307 
photoed stored in the internal memory in an electronic camera 307, i.e., image data, and 
attribute information is explained using drawing 6 . 

[0040] Although a photograph is taken with the gestalt of this operation after a user sets 
an electronic camera 307 to "panorama image photography mode" in case an image is 
photoed with an electronic camera 307 as mentioned above, the identifier which shows an 
one -set panorama image is automatically recorded by making it this photography mode 
into the attribute information on the photoed image which is recorded on an internal 
memory. 

[0041] As shown in drawing 6 , the image managed table 81 is placed into an internal 
memory, it has the image data storage field 82 and the attribute information storing field 
83, and attribute information is referred to with the photoed image data which corresponds 
for every image. 

[0042] here " the image data storage field 82 - image data 82a and 82b format data 
(NEITIBU data) original with an electronic camera 307 - or it is stored with either of the 
general-purpose format data, such as JPEG, and a user can choose these either according 
to a photography situation, and can store now. In addition, NEITIBU data are data 
obtained by A/D [ the output from CCD ] only, and although the time amount which record 
takes is short, they have come to be able to enlarge data size generally. Moreover, JPEG 
data can make data size small, although the time amount which record takes unlike 
NEITIBU data is taken. 

[0043] File names 84a and 84b, file types 85a and 85b, the photography time 86a and 86b, 
and the photography modes 87a and 87b are recorded on the attribute information storing 
field 83 as attribute information. 
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[0044] Here, file names 84a and 84b are unique file names which an electronic camera 307 
attaches automatically, 

[0045] It is shown whether file types 85a and 85b are other general-purpose formats which 
it being a NEITIBU data format or being a JPEG format or an electronic camera 307 
supports. 

[0046] The time and time amount at the time of the shutter release by wliich the 
photography time 86a and 86b was clocked by the calender and timer of the electronic 
camera 307 interior being pushed are recorded. 

[0047] The photography modes 87a and 87b show the photography mode chosen among 
some kinds of photography modes which an electronic camera 307 has at the time of 
photography, and when this is in "panoramic exposure mode", Identifiers 88a and 88b are 
added further. 

[0048] The number-of- sheets information 90a and 90b which shows the how many sheets 
in the modes 89a and ID 89b which are the unique numbers set when these identifiers 88a 
and 88b are set to panoramic exposure mode, and the mode of those they are is stored. 
Therefore, it will be said that two or more images which have the same modes 89a and ID 
89b in panoramic exposure mode are one set. In addition, in the example of drawing 6 , 
since scenery is photoed as an image of two right and left, it is the ID mode in which Modes 
89a and ID 89b are the same. 

[0049] With this equipment, although image data and attribute information are stored in 
an electronic camera 307 as mentioned above, further, an electronic camera 307 is 
connected to the body 301 of a computer, and the image + attribute information in an 
internal memory is copied to HD515. 

[0050] Next, a means to copy the data recorded in the electronic camera as mentioned 
above to a computer is described. 

[0051] Drawing 7 shows the screen when copying the data in an electronic camera 307 to 
HD515. 

[0052] This screen is displayed on a display 302 by the image data management system 
501, and a window 91 and a window 92 are displayed. 

[0053] The data of the internal memory in the electronic camera 307 called a camera 
catalog are displayed on a window 91, and the contraction image (thumbnail image) 94 and 
the attribute information-display field 95 of image data are displayed on it. Moreover, the 
firame 93 which indicates that the window 91 was chosen when an image was chosen by the 
user is displayed. 

[0054] The file name in attribute information, a file type, etc. are displayed on the attribute 
information-display field 95. In addition, it can be changed now by user assignment how 
far it displays among this attribute information. 

[0055] The data called the user catalog which are some image databases of the user who 
exists in HD515 are displayed on a window 92. Moreover, a copy wiU be performed, if a 
user chooses an image firom windows 91 and, as for a window 92, operates Drag&Drop. 
[0056] In addition, it changes [ (the data in an electronic camera 307 are eliminated), and ] 
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by assignment of a user whether it is a copy and whether they are (data remain in an 
electronic camera 307), and migration to both at this time. Moreover, in the midst of this 
copy actuation, NEITIBU data are changed into a predetermined general-purpose format 
with the image data management system 501, and if there is an image further photoed by 
the panorama image management tool 517 in panoramic exposure mode, those composition 
will be performed if needed. 

[0057] Next, it desciibes what kind of DS the data copied as mentioned above have in the 
user catalog. 

[0058] Drawing 8 shows the DS on the user catalog displayed on a window 92. 
[0059] A user catalog attaches the ID number of a proper about the image data stored in 
the interior with the image data management system 501, and is managed. That is, 
correspondence with the image data and attribute information which are linked to the ID 
number is taken, and, thereby, the probability of the base of management is carried out. 
[0060] Moreover, for every user catalog of this piece, although, as for a user catalog, a user 
can have the number of arbitration, as shown in drawing 8 , the catalog table 1100 is 
prepared. 

[0061] Here, in this catalog table 1100, the data ID 1101 of the image belonging to this 

catalog and the group ID 1102 of the group who belongs are held. 

[0062] Moreover, group ID 1102 links with the group attribute table 1103. 

[0063] This group attribute table 1103 is fundamentally the same as the catalog table 1100, 

and has the group ID belonging to the data ID 1105 or this group of the image belonging to 

this group. In addition, the places where the difference from the catalog table 1100 has the 

group attribute data 1104 in the head differ. 

[0064] As for this group attribute data 1104, a group name 1106, the date and time of 
creation 1107, and a group type 1110 are stored. 

[0065] When the identifier of the arbitration which the user attached attaches a group 
name 1106 and a group is made as a set of a panorama image, this group name is attached 
with a "panorama image" by the default. 

[0066] As for the date and time of creation 1107, time when a group is created is stored. 
[0067] When a user creates a group type 1110 and a group is made as "user creation" and a 
set of a panorama image, the information of a "panoramic exposure" enters. However, it 
links with an identifier further at the time of a panorama image, and mode ID89a is 
dedicated. 

[0068] Moreover, actual image data and attribute information are stored on a user catalog 
with the same structure as the image managed table 81 of above-mentioned drawing 6 . 
That is, these are referred to from the data control table 1108, further, it is linked to data 
ID 1109 within the data control table 1108, and correspondence with each image data and 
attribute information is taken. 

[0069] As mentioned above, in this equipment, it has the function in which a user does 
[ the image data in a user catalog 1 a category division for two or more images as one group. 
Namely, thereby, the inside of one user catalog is hierarchized and data can be managed 
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[0070] Next, three kinds of synthetic modes concerning the panorama image composition 
processing by the panorama image composition means 517 are explained. 
[0071] With this equipment, an electronic camera 307 is connected to the body 301 of a 
.computer, and in case the image + attribute information in an internal memory is copied to 
HD515, this attribute information is checked by the image data management system 501. 
Here, from that to which the identifier in panorama image photography mode exists in the 
attribute information on a user catalog, an one set image is extracted automatically, next 
processing of panorama image creation is started. However, since it has two or more 
synthetic modes of image composition processing in this invention, next synthetic mode is 
chosen. 

[0072] When the number of images is two, this synthetic mode with the full auto 
composition which compounds with perfect automatic at the time of three or more sheets 
When corresponding points are not enough called for in the check phase of the auto 
composition which has a user specify only the relative position of the four directions of an 
image and these full auto composition, or auto composition, Or when a user wants to save 
the time amount which detection of corresponding points takes, and to perform synthetic 
processing by shorter time amount, it consists of semi auto composition to which 
composition is carried out because a user specifies a great portion of overlap location: 
[0073] Here, one set of a panorama image is extracted and full auto composition is the 
mode chosen when it is two sheets. In this full auto composition, since four kinds of four 
directions can be considered as a synthetic location of the image of two sheets as shown in 
drawing 9 thru/or drawing 12 , processing which asks for the corresponding points of the 
part overlapped in [ these 1 four is performed, and the place where a lot of corresponding 
points which are in agreement more than predetermined level were able to be found is 
compounded as a right composition location. In addition, in this synthetic mode, as 
actuation of a user, it is only the actuation which copies an image into a computer from an 
electronic camera 307, and the panorama composition means 517 will do the rest 
automatically. Moreover, since two sheet composition is usually considered to be most 
except a special application, this synthetic processing is performed in most cases. 
Furthermore, in the gestalt of this operation, since it can say that certainty is low when the 
number of the coiresponding points which are in agreement more than predetermined 
level in any [ four ] case when performing processing which asks for corresponding points is 
below a predetermined amount, at this time, fiill auto composition processing is closed and 
it shifts to semi auto composition processing. 

[0074] Auto composition is the mode which one set of a panorama image is extracted, and 
is chosen when it is three or more sheets. And in this auto composition, it displays on the 
window as a user interface as shows an one-set image to drawing 13 and 14. All the images 
that belong to the group of a panorama image on a screen are first resized and displayed on 
the magnitude included in a window, a user looks at this, it Drag(s) in right sequence, only 
the physical relationship of the upper and lower sides and right and left is directed in this 
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window, and it rearranges into it. For example, in the example of drawing 14 , since the 
image located under a window 1401 becomes the rightmost in fact, if it Drag(s) on 
left-hand side, the panorama image composition means 517 will detect that it is the 
panorama horizontally located in a line three from the location. And as it has gone into the 
window, it is resized again and displayed like a window 1402. That is, 
corresponding-points extract processing in which it asks for the corresponding points of 
each image is directed based on a user. In addition, in the gestalt of this operation, when 
the number of the corresponding points which are in agreement more than predetermined 
level with the panorama image composition means 517 in the case of the 
corresponding-points extract processing by auto composition is more than a predetermined 
amount, it compounds as a right composition location, otherwise, since certainty is low, 
auto composition processing is closed, and it shifts to semi auto composition processing. 
[0075] Semi auto composition is the mode chosen when a user wants to save the time 
amount which a corresponding-points extract takes, and to obtain a synthetic result early 
more in full auto composition or auto composition processing when the certainty of a 
corresponding-points extract is low or. And in this semi auto composition, a user drag(s) 
the image displayed on the window as a user interface as shown in drawing 15 and 
drawing 16 , and specifies the location of the great portion of overlap. That is, in this semi 
auto composition, based on [ time / of auto composition processing ] the positional 
information specified by the user, corresponding-points extract processing in which it asks 
for the corresponding points of each image performs corresponding -points extract 
processing in the much narrow range, asks for a location most in agreement, and performs 
synthetic processing further from this result. For example, in the example of drawing 16 , 
as shown in a window 1801, all the images that belong to the group of a panorama image 
on a screen are first resized and displayed on the magnitude included in a window 1801, 
and a user looks at this, and as shown in a window 1802, a great portion of overlap location 
is doubled, and it piles up. Since the place piled up mutually displays by carrying out an 
AND operation for every bit per pixel, both images are transparent and the overlapping 
part is visible. And it is resized again and it is displayed that it has gone into the window. 
In addition, the actuation by the semi auto composition on this window is fundamentally 
the same as actuation by auto composition, and there are few burdens of a user. A 
di£Ference is only the point (semi auto composition processing) of overlapping that (auto 
composition processing) which rearranges and displays the location which Drag(ed) and 
detached the image with pointing devices, such as a mouse, only using the information on 
physical relationship, using the information on a location as it is, and displaying. Moreover, 
since it is transparent and visible by the previous AND operation also in Drag, a great 
portion of location can be put together easily. 

[0076] In addition, after corresponding-points extract processing finishes in the case of 
which [ of the above-mentioned S3mthetic mode ] and the overlap range of an image is 
called for, the way of seamless processing is changed and synthetic processing is performed 
so that it may distinguish whether it is an alphabetic character subject's manuscript image. 
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and whether it is the usual natural drawing, next may mention later based on this 
distinction result. Moreover, although a user performs actuation for a corresponding-points 
extract as mentioned above in auto composition and semi auto composition, since it is only 
actuation of both drag(ing) an image and is common actuation that this is the simplest and, 
a user's burden is small. Moreover, since an image is drag(ed) and it only doubles with a 
great portion of location at the time of semi auto composition, it is much easy and easier 
than actuation of specifying the point of the conventional example clearly . 
[0077] Next, actuation of the gestalt of this operation is explained according to the flow 
chart of drawing 2323 thru/or drawing 3 1 , referring to drawing 17 thru/or drawing 22 . 
[0078] First, the actuation at the time of copying the image data in an electronic camera 
into a computer is explained. 

[0079] Drawing 23 is the flow chart of the processing performed in case the image data in 
an electronic camera is copied into a computer. In addition, in the flow chart of drawing 23 , 
especially, as long as there is no notice, the image data management system 501 performs 

the processing actuation. 

[0080] In this drawing, in order that a copy may process to all required images first, it 
judges whether processing has finished to all images (SlOOO). If processing has finished, 
processing is advanced to the below -mentioned step S1009, and if it has not finished yet, 
processing will be advanced to the following step SlOOl. 

[0081] First, the attribute information which accompanied the image data of a piece and it 
first is acquired in copy actuation (SlOOl): From the file types 85a and 85b within attribute 
information, this image data judges whether it is NEITIBU data (S1002). If it is not 
NEITIBU data, processing will be advanced to the below-mentioned step S1004. Moreover, 
NEITIBU data will be changed into the general-purpose formats (JPEG, TIFF, etc.) 
decided as a default if it is NEITIBU data (S1003). And if conversion finishes, file types 
85a and 85b will also be updated. 

[0082] Next, it confirms whether to be the image which investigated the photography 
modes 87a and 87b, and was photoed in panoramic exposure mode (S1004). When it is not 
a panorama image, it registers as usual image data (S1008). The data ID of a proper are 
specifically attached and registered into the data control table 1108 in drawing 8 , and the 
data ID is registered into the catalog table 1100. 

[0083] When it is a panorama image, it confirms whether to be finishing [ the group for 
these panorama images / creation ] already (S1005): This follows the catalog table of 
drawing 8 , and when mode ID89a of group ID looks at whether it is the same as mode 
ID89a of an image, it is performed. 

[0084] A group is created when there is no corresponding group (S1006). This newly 
registers group ID 1102 into the catalog table 1100, and creates a group name 1106, the 
date and time of creation 1107, and a group type 1110. It is recorded on a group type 1110 
as a "panoramic exposure'*, and mode ID89a in the attribute information on an image is 
dedicated. 

[0085] And the data ID of a proper are attached to this panorama image data, and it 
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registers with the managed table 1108, and registers with data ID 1105 (S1007). 
[0086] It carries out from the above step SIOOO to all the images that copy a series of 
processings of S 1008. If processing finishes to all images, although copied until now, if it 
confirms whether the group of a panorama image was made (S1009) and it is made, the 
panorama image composition means 517 will perform in inside panorama image 
composition processing which mentions later using the image in a group (SlOlO). If there is 
nothing, processing will be ended now. 

[0087] Next, explanation about the panorama image composition processing in said step 
S 10 10 is given. 

[0088] Drawing 24 shows the flow chart of panorama image composition processing. In 
addition, in the flow chart of drawing 24 , especially, as long as there is no notice, the 
panorama image composition means 517 performs the processing actuation. 
[0089] In this drawing, the panorama image composition means 517 confirms first whether 
there are more images in a group than two sheets and two sheets (S1200). At the time of 
two sheets, the full auto composition processing mentioned later is started (81202). When 
[ than two sheets ] more, the auto composition processing mentioned later is started 
(S1201). After each processing at these steps S1201 and S1202 finishes, the result of that 
synthetic processing checks a success or failure (S1203, S1204). Decision of this success or 
failure is judged by the abflity of the point with which images correspond to have been 
found enough. Therefore, in the synthetic whole processing, since a judgment is made in an 
early phase, failure of the time amount for which it waits until a user obtains the result is 
also short, and a result can be managed. And if it is a success, processing is termination, 
and processing is ended after performing semi auto composition processing mentioned 
later (S1205), if it is failure. 

[0090] Next, explanation about the full auto composition processing in said step S1201 is 
given. 

[0091] Drawing 25 shows the flow chart of auto composition processing. In addition, in the 
flow chart of drawing 25 , especially, as long as there is no notice, the panorama image 
composition means 517 performs the processing actuation. 

[0092] In this drawing, the panorama image composition means 517 acquires first the 
information on the physical relationship which the user rearranged (Si 301). Next, the 
range searched although corresponding points in agreement are found, i.e., the matching 
range, is set up (S1302). In addition, as the Ruhr when taking a photograph as a panorama 
image, a minimum of 10%, if making it overlapping a maximum of 50% and the gap of a 
direction which carries out a right angle to it are decided to be 5% or less, respectively, as 
shown in drawing 17 , the surely overlapped range will turn into range shown in the slash 
area 1504 of the left image 1501 of drawing 17 . Moreover, the range which may be 
overlapped turns into range shown in the slash area 1505 of the right image 1502. Now, 
when the point shown in the point 1503 all over the area of the slash area 1504 follows the 
above-mentioned Ruhr, it will have a point [ be / it / under / area / which is shown in the 
search range 1506 in the area of the slash area 1505 / correspondence ]. It will be found in 
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the corresponding-points extract processing mentioned later whether it matches to this 
area. 

[0093] A set of the parameter which will be used for the entry which more than searchs at 
said step S1302 if it returns to a flow chart performs processing which extracts 
corresponding points next (S1303). In addition, about the detail of this 
corresponding-points extract processing, it mentions later. Since the number of the 
corresponding points which were able to be found was not able to judge whether it was 
beyond a predetermined value (N individual) (S1304) and corresponding points were not 
able to be enough found automatically at the time of below a predetermined value after 
corresponding-points extract processing finished, it progresses to semi auto composition 
processing. When [ than a predetermined value ] more, it progresses to synthetic 
parameter setup processing (S1305). In this synthetic parameter setup processing, it asks 
for the parameter of the migration used in the case of composition, expansion (contraction), 
and rotation from the coordinate of previous corresponding points. In addition, this detail 
is mentioned later. Finally, image composition processing is performed based on these 
parameters (S1306). This detail is also mentioned later. 

[0094] Next, explanation about the full auto composition processing in said step S1202 is 
given. 

[0095] Drawing 26 shows the flow chart of full auto composition processing. In addition, in 
the flow chart of drawing 26 , especially, as long as there is no notice, the panorama image 
composition means 517 performs the processing actuation. 

[0096] In this drawing, although the panorama image composition means 517 performs a 
matching entry first (S1601), as for this, said step S1302 and processing are the same. 
[0097] Next, corresponding-points extract processing is performed 4 times. Since number of 
sheets is limited to two sheets in full auto composition processing, an image 1 and the 
image 2 of the physical relationship considered are four l^inds of right and left and right 
and left the upper and lower sides and on the bottom. Then, corresponding-points extract 
processing is carried out about these four cases, and the numbers currently extracted as 
corresponding points, respectively and such average coincidence level are held. These 
processings are processings from step S1602 to 81609. 

[0098] And it is confirmed whether, in [ four ] the upper case, there are some which have 
extracted the corresponding points beyond a predetermined value (N individual) (S1610). 
If there is no one, it will progress to semi auto composition processing. If it is, suppose in it 
that it is the best thing of average coincidence level true physical relationship (81611). In 
addition, although it should become only one of four by the usual image when 
corresponding points are beyond predetermined values, a character which was alike, for 
example when a picture of a manuscript etc. was divided and taken is located in a line, and 
beyond a predetermined value can be extracted as corresponding points also in the time of 
the physical relationship which is not right. Then, what fits most (what has the highest 
average coincidence leveD is chosen at this step S1611. 

[0099] Although it progresses to the next synthetic parameter processing (S1612) and 
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image composition processing (S1613) after processing of step S1611 finishes, this is the 
same processing as said steps S1305 and S1306, and is mentioned later for details. 
[0100] Next, explanation about the semi auto composition processing in said step S1205 is 
given. Processing here is almost the same as that of the time of auto composition 
processing. 

[OlOl] Drawing 27 shows the flow chart of semi auto composition processing. In addition, 
in the flow chart of drawing 27 , especially, as long as there is no notice, the panorama 
image composition means 517 performs the processing actuation. 

[0102] In this drawing, the panorama image composition means 517 acquires first the 
overlap positional information which the user of an image doubled (S1701). And although 
the matching range is set up (Si 702), the range here turns into predetermined range 
(error-range + margin of the location which the user assumed doubled). Therefore, it 
becomes the range much narrower than the range at the time of auto composition 
processing, and compaction of computation time and improvement in precision are 
achieved. 

[0103] Although it progresses to the next corresponding-points extract processing (S1703), 
synthetic parameter setup processing (S1704), and image composition processing (S1705) 
after processing of step S1702 finishes, each of these is the same as that of the time of auto 
composition processing. 

[0104] Next, explanation about corresponding-points extract processing is given. 
[0105] First, the outUne of corresponding-points extract processing is described with 
reference to drawing 18 . 

[0106] Drawing 18 shows the left in the case of a corresponding-points extract, and the 
example at the time of two right images here. When two number of sheets of an image is 
large, since what is necessary is just to repeat composition of two sheets several times, as 
processing, it is fundamentally the same. 

[0107] First, the range 2005 which foUows the Ruhr at the time of photography and sets up 
a template is set as the 90% long and 10% wide range. Moreover, the range to search is the 
range of possibility that the point of corresponding exists, and is set as 100% long and the 
50% wide range 2006. From the area of the template setting range 2005 in an image, an 
edge looks for a strong point beyond a predetermined value, and starts length and a n 
pixels wide rectangle as a template image 2003 centering on there. This template image 
2003 is placed on the search range 2004, and that difference is taken per pixel. It shifts 1 
pixel of the search range 2004 tops at a time, and asks for the place where this sum total 
serves as min. If the minimum value of the result of having searched all search range tops 
is below a predetermined value, it will hold the points (x y), and (x*, yO of those as a pair of 
corresponding points. 

[0108] Although the above processing serves as an outline of corresponding-points extract 
processing, this is explained once again along with the flow chart of drawing 28 . 
[0109] Drawing 28 shows the flow chart of corresponding-points extract processing. In 
addition, in the flow chart of drawing 28 , especially, as long as there is no notice, the 
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panorama image composition means 517 performs the processing actuation. 
[Olio] In this drawing, the panorama image composition means 517 creates an edge 
extract image first (S1901). And an edge looks for the powerful point beyond a 
predetermined value firom the range 2005 which sets up the template in this edge extract 
image (S1902). And if there is the point, an image will be cut down with the rectangle of 
every **n pixels of every direction firom the point, and it wiU consider as a template image 

(51903) . 

[0111] Next, the search range in a right image is set up from the location of the point 

(51904) . And the difference of a pixel value takes an absolute value for the image and 
template image in the search range per superposition and pixel, and the sum tota:l is 
searched for (S 1905). 

[0112] Moreover, it confirms whether to be the minimum value till then (S1906), and if the 
total value of this difference is so, it will hold the coordinate arid the minimum value of the 
point in that search range (S1907). The above is repeated in all search range and a point (it 
has the minimum difference) most in agreement is found out. 
[0113] And it checks (S1908), and the minimum value calculated as a result is a 
sufficiently small value, or (are they clear corresponding points?) it judges whether all 
search range was searched as compared with the predetermined value L (S1909); When 
smaller than the predetermined value L, the coordinate (x y) of the point which cut down 
the template image on the list of corresponding points, the coordinate (x*, yO, and the value 
of the minimum value of the point with which the minimum value was calculated are 
registered (S1910). 

[0114] If the above is performed to all template setting ranges (S1911) and it ends, the 
average value will be calculated from all the minimum values under list of corresponding 
points, and this is held as a coincidence level value (S1912). Corresponding-points extract 
processing is ended above. 

[0115] Next, synthetic parameter setup processing is explained. When an image is made 
into two sheets, (also in composition of two or more sheets, your thinking by two sheets 
first of aU, since it is the repeat of composition of two sheets), and its gap can be expressed 
with advancing side by side of x and the direction of y , rotation, and the difference of a 
dilation ratio. Therefore, a point (x y), and corresponding (corresponding x\ yO can be 
expressed as follows. 
[0116] 

[Equation l] 



15/24 



Japanese Publication number : 09-093431A 



Here, in theta, the angle of rotation of the circumference of the Z-axis, and deltax and delta 
y show advancing side by side, and m shows a scale factor. Therefore, this coordinate 
transformation can be expressed by asking for Parameters A, B, C, and D. In previous 
corresponding-points extract processing, corresponding points (x y) and two or more groups 
of (x', yO were acquired. This is asked for Parameters A, B, C, and D using a least square 
method. 

[0117] That is, it is the conditions of epsilon=sigma [{(Ax+By+C) •x'}2+{(-Bx+Ay+D) -y'}2l 
•>min, and is [0118]. 
[Equation 2] 



It asks for the ******** parameters A, B, C, and D. 

[0119] It is here and is Pi =sigmax2+sigmay2P2 =sigmaxP3 =sigmayP4 
=sigmaxx'+sigmayy'P5 =sigmaxy'-sigmax'yP6 =sigmax'P7 =sigmayT8 =n (the number of 
corresponding points). 

If it carries out. Parameters A, B, C, and D can be expressed as follows. 
[0120] 

[Equation 3] 



Parameters A, B, C, and D are computed by calculating PS from this parameter PI, and 
substituting for an upper formula. 

[0121] Next, image composition processing is explained. 

[0122] Drawing 29 shows the flow chart of the synthetic whole processing. In addition, in 
the flow chart of drawing 29 , especially, as long as there is no notice, the panorama image 
composition means 517 performs the processing actuation. 

[0123] In this drawing, the panorama image composition means 517 performs the entry at 
the time of performing the overlap range, setup of Rhine to connect, and seamless 
processing first (S2901). 
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[0124] About a setup of this overlap range, the formula and x'=Ax+By+Cy'=-Bx+Ay+D 
which were calculated in the top are used. 

[0125] For example, if the image 3004 located in the left considers as the image of 640x480 
dots as shown in drawing 19 , (x2', y2') which substituted a coordinate (639 0) and 
(639,479) for the upper formula (x y), and were obtained (xl', y lO will become the overlap 
range of the right image 3005. From here, the overlap range 3002 can be found as a 
coordinate location of a left image, and it asks as Rhine which Rhine 3001 of the middle of 
this range connects. The range at the time of performing seamless processing is set up in 
quest of the field 3003 of the predetermined width of face set up beforehand firom the 
center of the overlap range. At this time, when the width of face of the overlap range is 
narrower than predetermined width of face, it has the width efface of the overlap range, 
and considers as a field 3003. 

[0126] Next, histogram processing which distinguishes whether it is an alphabetic 
character subject's image and whether it is a natural image is performed (S2902). 
[0127] First, the histogram of brightness is created about the pixel of overlap within the 
limits 3002. Since the same result only comes out at this time even if it performs to both 
images, it is easy to come out only to image (for example, left-hand side image) of one of the 
two. Therefore, a histogram setting range is narrow, and since it is only image of one of the 
two, the time amount which histogram processing takes is very short, and ends. 
[0128] Here, drawing which distinguishes whether it is the image of the alphabetic 
character subject who used the histogram of brightness for drawing 20 and drawing 21 , 
and whether it is a natural image is shown. As shown in these drawings, the shaft of the 
brightness of a histogram is roughly trichotomized and it asks for the total frequencies bl, 
b2, and b3 of each range al, a2, and a3. And bl is larger than a threshold thl (the part 
considered to be an alphabetic character included more than the constant rate), and b2 is 
smaller than a threshold th2 (brightness considered not to be an alphabetic character or 
paper, either has few things than a constant rate), and when larger (the part considered to 
be paper is included more than the constant rate) than a threshold th3, it is judged as an 
alphabetic character image and judged [ b3 carries out, and ] as a natural image except it. 
[0129] After finishing histogram processing at step S2902 as mentioned above next, this 
distinguishes whether it is an alphabetic character image (S2903), synthetic processing 
(S2904) which does not include seamless processing by this distinction result, or synthetic 
processing including seamless processing is performed (S2905), and processing is ended. 
[0130] Next, the image composition processing which does not include seamless processing 
of said step S2904 is explained. 

[0131] First, the outline of the image composition processing which does not include 
seamless processing with reference to drawing 22 is described. 

[0132] What illustrated this to drawing 22 is shown. When images are the left image 2101 
and the right image 2102, the image 2101 twice the magnitude of the left is secured as a 
synthetic image field 2103. The field to Rhine 3001 which connects the left image 2101 first 
. is copied to this sjnithetic image field 2103 as it is. 
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[0133] next the remaining fields (x y) of a synthetic image *" formula 
x*=Ax+By+Cy--Bx+Ay+D - corresponding (x', yO • it asks. And the pixel of (x*, y') of a 
right image is copied to (x, y). This is performed to all the remaining fields of the synthetic 
image field 2103. 

[0134] Drawing 30 shows the flow chart of the image composition processing which does 
not include seamless processing. In addition, in the flow chart of drawing 30 R> 0, 
especially, as long as there is no notice, the panorama image composition means 517 
performs the processing actuation. 

[0135] First, a twice as many field as the 1st image (left image in drawing 22 ) is secured as 

a synthetic image field (S2201). And the field on the left of Rhine 3001 of the 1st image is 

simply copied to this synthetic image field (S2202). 

[0136] next " the remaining fields (x y) of a synthetic image - formula 

x -Ax+By-i-Cy'=-Bx+Ay+D - corresponding (x*, yO - it asks (S2203). And it confirms 

whether (x*, y') are in the 2nd image (right image in drawing 22 ) (S2204), and if there is 

nothing, processing is advanced to the below-mentioned step S2206, and if it is, the pixel of 

(x\ yO will be copied to (x, y) (S2205). 

[0137] Furthermore, it judges whether processing from the above step S2203 to S2205 was 
repeatedly performed to all the remaining fields of a synthetic image (S2206), and if it goes 
to all fields, processing will be ended now. 

[0138] Next, the image composition processing including seamless processing of said step 
S2905 is explained. 

[0139] Drawing 31 shows the flow chart of the image composition processing including 
seamless processing. In addition, in the flow chart of drawing 31 , especially, as long as 
there is no notice, the panorama image composition means 517 performs the processing 
actuation. 

[0140] First, a twice as many field as the 1st image (left image in drawing 22 ) is secured as 
a synthetic image field like said step S2201 (S3201). Next, the field on the left of the range 
3003 which performs seamless processing is simply copied to a synthetic image field 
(S3202). 

[0141] And seamless processing is performed (S3203). That is, first, a corresponding 
coordinate is searched for by formula x'=Ax+By+Cy'=-Bx+Ay+D, and it asks for the pixels 
pi and p2 to which both images correspond. And synthetic pixel p3=(l-a) *p l+a*p2a = the 
width of face of the location / the seamless range of the x directions of a pixel performs 
seamless processing within seamless limits. 

[0142] After finishing seamless processing at step S3203 next, the remaining fields of the 
2nd image are copied to a synthetic image field (S3204), and processing is ended. In 
addition, this way is the same as that of processing fi-om step S2203 of drawing 30 to S2206. 
Above, a final panorama composition image can be created. 

[0143] As explained above, the problem of the double counterpart in an alphabetic 
character can be solved by not performing seamless processing, when according to the 
gestalt of this operation the image to compound distinguishes whether it is belonging to an 
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alphabetic character subject and is judged to be an alphabetic character subject in advance 
of panorama composition processing, and the problem in a panorama composition image 
that a knot will be conspicuous can be solved. 

[0144] (Gestalt of the 2nd operation) Next, the gestalt of the 2nd operation by this 
invention is explained. 

[0145] As a gestalt of the 2nd operation, how to set Rhine connected at the time of an 
alphabetic character image as Rhine with least (for brightness to be high) focus in the 
overlap range is explained below. 

[0146] In addition, the configuration in the gestalt of this 2nd operation and actuation are 
the same as that of the gestalt of the 1st operation fundamentally, and only the places 
which ask for Rhine which 3001 in drawing 19 connects differ. Therefore, here explains 
only a different part &om the gestalt of the 1st operation. 

[0147] With the gestalt of said 1st operation, with the panorama composition means 517, 
although the location of lamination was made into Rhine of middle, as shown in drawing 
32 , Rhine 2801 with least focus is looked for among the width efface 2802 of lamination in 
overlap which can be taken, and lamination is performed by the gestalt of this operation, 
there. Thereby, synthetic processing in which a knot is not further conspicuous can be 
performed. 

[0148] Next, actuation with the gestalt of this operation is explained. 

[0149] The flow chart of processing of the characteristic part in the gestalt of this operation 
is shown in drawing 33 . In addition, this flow chart explains as a case of composition on 
either side. However, in the upper and lower sides, you may think the same way. 
[0150] In this drawing, the panorama image composition means 517 initializes the variable 
max holding maximum to 0 first (S3301). And all the pixels belonging to vertical Rhine are 
acquired firom the leftmost side of the overlap range shown by 3002 of drawing 19 (S3302). 
[0151] Next, it substitutes for Variable Sum in quest of total of these pixel values (S3303). 
in addition since an image is an alphabetic character image here " paper - white - or 
other bright colors it is - an alphabetic character black or it can be said that they are 
other dark colors. Then, it asks for total of a pixel for every Rhine, and it can be said that it 
is rare for Rhine of the largest value in it to divide an alphabetic character most. Moreover, 
although the filter of 3x3 may be let pass, calculated and determined in order to 
investigate the number of an alphabetic character edge for every Rhine for example, the 
simple addition using the description of a previous alphabetic character image of the 
processing time may be shorter. 

[0152] And it judges whether Variable Sum is larger than Variable max (S3304), and if 
Variable Sum is larger than Variable max, it will be said that there are few parts of an 
alphabetic character, and Variable Sum will be substituted for Variable max and will be 
held (S3305). Moreover, the location pos of Rhine is also held. 

[0153] When the above is performed to the rightmost side of the overlap range one by one, 
the location pos currently held will call it Rhine with few parts of an alphabetic character. 
Rhine of this location is set up as Rhine 3001 in drawing 19 , and subsequent processings 



19/24 



Japanese Publication number : 09-093431 A 



are continued. 

[0154] According to the gestalt of the 2nd operation, Rhine with least focus is looked for 
among the above width of face of the lamination which can be taken in overlap, and 
lamination is performed there. Thereby, synthetic processing in which a knot is not further 
conspicuous can be performed. 

[0155] (Gestalt of the 3rd operation) Next, the gestalt of the 3rd operation by this invention 
is explained. 

[0156] As another means which is below different from the conventional seamless 
processing as a gestalt of the 3rd operation, composition performs lamination, shading-off 
processing is added along with the stuck boundary part, and how to erase a knot is 
explained. 

[0157] In addition, only the contents of the synthetic processing which the configuration in 

the gestalt of this 3rd operation and actuation become unnecessary [ the judgment part of 

the brightness histogram shown by drawing 29 R> 9 ] although it is the same as that of the 

gestalt of the 1st operation fundamentally, and is shown by drawing 19 differ. Therefore, 

here explains only a different part from the gestalt of the 1st operation. 

[0158] The panorama image composition means 517 performs shading-off processing by 

the gestalt of this operation, as shown in drawing 34 and drawing 35 . 

[0159] That is, with the gestalt of this operation, as shown in drawing 34 , the 1st image 

3503 adds shading-ofif processing to the edge part 3501 as it is, using all as some synthetic 

images. Furthermore, as a filter of shading-off processing, it carries out using the matrix 

3502 of 3x3 as shown, for example in drawing 35 . 

[0160] Next, the actuation in the gestalt of this operation is explained. 

[0161] The flow chart of the image composition processing in the gestalt of this operation to 
drawing 36 is shown. 

[0162] In this drawing, the panorama image composition means 517 secures a twice as 
many field as the 1st image (left image in drawing 34 ) as a synthetic image field first 
(S3401). And the 1st image is simply copied to this synthetic image field (S3402). 
[0163] next the remaining fields (x y) of a synthetic image " formula 
x'=Ax+By+Cy'=-Bx+Ay+D - corresponding (x\ yO - it asks (S3403). And it confirms 
whether (x*, yO are in the 2nd image (right image in drawing 34 ) (S3404), and if there is 
nothing, processing will be advanced to the below-mentioned step S3402, and the pixel of if 
it is (x', yO will be copied to (x, y) (S3405). 

[0164] Furthermore, if it judges whether processing from the above step S3403 to S3405 
was repeatedly performed to aU the remaining fields of a synthetic image (S3406) and 
there is no Une, in order to repeat processing to all the remaining fields of a synthetic 
image, processing is retiirned to said step S3403. 

[0165] Moreover, if it judges that it went by step S3406 to all fields, shading-off processing 
will be performed, covering the filter of 3502 to the edge part (3501 in drawing 3434 ) of the 
1st image (S3407), and processing will be ended now. 

[0166] Thus, although the conventional seamless processing is taking how composition 
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performing lamination, adding shading-ojEf processing along with the stuck part as another 
different means, and erasing a knot and some knot remains with the gestalt of the 3rd 
operation compared with previous seamless processing, there is an advantage to which the 
processing time can be managed in a short time. Moreover, the effectiveness of becoming 
unnecessary [ distinction called an alphabetic character subject's image and a natural 
image ] is acquired. 

[0167] (Gestalt of the 4th operation) Next, the gestalt of the 4th operation by this invention 

is explained. 

[0168] When printing a panorama image on below as a gestalt of the 4th operation later In 
the case of the printer (binary PURITA) by which it has a means by which a user specifies 
beforehand the printer which carries out a print, and the specified printer outputs binary 
[ which is printed by error diffusion methods, such as an ink jet printer (BJ) and a laser 
beam printer (LBP), ] How to perform lamination of an image in Rhine (Rhine where 
brightness is high) where concentration is low is explained. 

[0169] in addition , although the configuration in the gestalt of this 4th operation and 
actuation be the same as that of the gestalt of the 1st operation fiindariientally , when the 
binary printer print by the error diffusion method as a printer as mentioned above be 
specify , it be characterize by perform image composition bordering on Rhine where the 
brightness of the image in the duplicate image field be the highest , and the contents of the 
processing about this differ . Therefore, here explains only a different part firom the gestalt 
of the 1st operation. 

[0170] The flow chart which explains the whole actuation to drawing 37 is shown. 

[0171] First, the panorama image composition means 517 confirms whether the printer is 

assigned to the user (S3601), and it checks it in a binary printer (S3602). 

[0172] If it is a binary printer, processing explained henceforth will be performed (S3603). 

The printer will not be specified, or if it is not a binary printer, the gestalt of the 1st 

operation will be processed (S3604). 

[0173] In addition, although synthetic processing with the gestalt of this operation is 
fundamentally the same as the gestalt of the 1st operation, distinction of the alphabetic 
character image shown in drawing 29 or a natural image becomes unnecessary. And it 
differs firom the place in drawing 19 R> 9 which connects and asks for Rhine 3001. 
[0174] Drawing 33 is drawing also showing the flow chart of processing with the gestalt of 
this operation, and shows the processing actuation by the panorama image composition 
means 517 at the time of asking for the brightest Rhine firom the range which the image 
overlaps. However, this flow chart explains as a case of composition on either side. 
Moreover, in the upper and lower sides, you may think the same way. 
[0175] First, the panorama image composition means 517 initiahzes the variable max 
holding maximum to 0 (S3301). And all the pixels belonging to vertical Rhine are acquired 
fi-om the leftmost side of the overlap range 3002 shown by drawing 19 (S3302). It 
substitutes for Variable Sum in quest of total of these pixel values (S3303). It judges 
whether Sum is larger than max (S3304), and if Sum is larger than max, it will be called 
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brighter Rhine, and a value Sum will be assigned to max and will be held (S3305). 
Moreover, the location pos of Rhine is also held. When the above is performed to the 
rightmost side of the overlap range one by one, it will be called the brightest Rhine in the 
location pos currently held. Rhine of this location is set up as Rhine 3001 in drawing 19 . 
[0176] Next, the processing actuation at the time of performing actual synthetic processing 
by the panorama image composition means 517 is explained. Drawing 30 is drawing also 
showing the flow chart of processing with the gestalt of this operation. 
[0177] In this drawing, the panorama image composition means 517 secures a twice as 
many field as the 1st image (left image in drawing 22 ) as a synthetic image field first 
(S2201). And the field on the left of Rhine 3001 of the 1st image is simply copied to this 
synthetic image field (S2202). the remaining fields (x y) of a synthetic image - firom 
formula x'=Ax+By+Cy'=-Bx+Ay+D - corresponding (x\ yO " it asks (S2203). 
[0178] Next, the pixel of if check (S2204) and it is [ whether (x', y') are in the 2nd image 
(right image in drawing 22 R> 2) and ] (x\ yO is copied to (x, y) (S2205). Processing ends the 
above repeatedly to aU the remaining fields of a synthetic image. 

[0179] It has a means to specify the printer in the above cases of printing a synthetic image 
according to the gestalt of the 4th operation, and, in the case of the binary printer by which 
the printer specified as the user outputs BJ and binary [, such as LBP, ], Rhine to connect 
is made into Rhine (Rhine where concentration is the lowest) where brightness is the 
highest. Thereby, in the case of a binary printer, outputting by the error diffusion method 
is common, and since a printing dot diffuses the low place of concentration most at this 
time, the efiectiveness that a knot is not conspicuous is induced. Moreover, most printers 
in current o£6ce and a current home are binary printers, such as said BJ, LBP, etc., and it 
is thought further [ when / most / a user uses a printer clearly ] that this processing is 
applicable. 
[0180] 

[Effect of the Invention] By distinguishing whether it is the image with which the image of 
the image field which overlapped by having considered as the configuration mentioned 
above according to this invention makes an alphabetic character a subject, as explained in 
fuU detail above, and performing image composition, after performing a different image 
processing according to this distinction result, the problem that where of the knot in a 
panorama composition image will be conspicuous can be solved, and good image 
composition is attained. 

[0181] Moreover, although the resolution of a part obscured by adding shading-off 
processing to the boundary part which connects two or more images as another means 
different from the conventional seamless processing, and performing image composition 
falls a little, the high-speed processing in which a knot is not conspicuous is realizable. 
[0182] Furthermore, by performing image composition bordering on Rhine where the 
brightness of the image in the duplicate image field is the highest when the binary printer 
printed by the error diffusion method as a printer which outputs a synthetic image is 
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specified, a printing dot diffuses the low place of concentration most in the case of a print, 
and, thereby, the effectiveness that a knot is not conspicuous can be acquired. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing ll It is the outline external view of this equipment. 

[Drawing 2] It is the block diagram showing the configuration of this equipment. 
[Drawing 3] It is drawing showing the case where the 1st example is applied in an 
alphabetic character image. 

[Drawing 4] It is drawing showing the histogram of the brightness in an alphabetic 
character image. 

[Drawing 5] It is drawing showing the histogram of the brightness in a natural image. . 
[Drawing 61 It is drawing showing the DS of the image data recorded in the electronic 
camera. 

[Drawing 7l It is drawing showing the screen at the time of copying the image data in an 
electronic camera. 

[Drawing 8l It is the DS when managing data within a computer. 

[Drawing 9] It is drawing showing the combination of the image of two sheets assumed at 
the time of full auto composition processing. 

[Drawing 10] It is drawing showing the combination of the image of two sheets assumed at 
the time of full auto composition processing. 

[Drawing 111 It is drawing showing the combination of the image of two sheets assumed at 
the time of full auto composition processing. 

[Drawing 12l It is drawing showing the combination of the image of two sheets assumed at 
the time of full auto composition processing. 

[Drawing 13] It is drawing showing the user interface of auto composition processing. 
[Drawing 14] It is drawing showing the user interface of auto composition processing. 
[Drawing 15] It is drawing showing the user interface of semi auto composition processing. 
[Drawing 16] It is drawing showing the user interface of semi auto composition processing. 
[Drawing 17] It is drawing showing the matching range at the time of comppsition. 
[Drawing 18] It is drawing showing the template image and matching range of 
corresponding-points extract processing. 

[Drawing 19] They are the overlap range and drawing showing a setup of Rhine to connect. 
[Drawing 20] It is drawing showing the histogram of the brightness for distinguishing an 
alphabetic character image or a natural image. 

[Drawing 2l] It is drawing showing the histogram of the brightness for distinguishing an 

alphabetic character image or a nattiral image. 

[Drawing 22l It is drawing illustrating synthetic processing. 

[Drawing 23l It is the flow chart of the processing performed in case the image data in an 
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electronic camera is copied. 

[Drawing 24] It is the flow chart which shows the flow in the whole case of panorama 
composition. 

[Drawing 25] It is the flow chart of auto composition processing. 

[Drawing 26] It is the flow chart of full auto composition processing. 

[Drawdng 27] It is the flow chart of semi auto composition processing. 

[Drawing 28] It is the flow chart of corresponding-points extract processing. 

[Drawing 29] It is the flow chart which shows processing actuation of the whole including 

distinction of being an alphabetic character image. 

[Drawing 30] It is the flow chart of synthetic processing. 

[Drawing 3l] It is a flow chart about the synthetic processing including seamless 
processing. 

[Drawing 32] It is drawing showing the case where the 2nd example is appUed in an 
alphabetic character image. 

[Drawing 33] It is a flow chart at the time of asldng for the brightest Rhine from the 
overlap range. 

[Drawing 34] It is drawing showing the shading-ofif processing in the 3rd example. 
[Drawing 35] It is drawing showing the filter used for shading-o£f processing. 
[Drawling 36] It is the flow chart of the synthetic processing in the 3rd example. 
[Drawing 37] It is the flow chart which shows processing actuation of the 4th whole 
example. 

[Drawing 38] It is drawing explaining seamless processing. 

[Drawing 39] It is drawing explaining seamless processing when overlap width of face is 
large. 

[Drawing 40l It is drawing showing the case where seamless processing is performed in an 
alphabetic character image. 
[Description of Notations] 

301 Body of Computer 

302 Display 

303 Mouse 

306 Genersd Interface 

307 Electronic Camera 
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